fix(store): allocate graph id before batch writes - #3153
Conversation
imbajin
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: The new batch graph-ID path can leave RocksDB transaction locks held on allocation errors, persist IDs for batches that never commit, and reuse an ID while an in-flight batch still owns encoded keys. The added regression test also exercises an invalid partition instead of the empty-table transaction path. Evidence: exact-head static review; mvn test -pl hugegraph-store/hg-store-test -am -P store-core-test -Djacoco.skip=true -ntp passed 3 tests locally; latest-head CI completed, with Codecov upload 429 treated as non-blocking.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #3153 +/- ##
============================================
- Coverage 39.23% 37.41% -1.82%
- Complexity 264 6447 +6183
============================================
Files 771 800 +29
Lines 65938 68653 +2715
Branches 8759 9106 +347
============================================
- Hits 25872 25688 -184
- Misses 37310 39951 +2641
- Partials 2756 3014 +258 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
- assert the truncated graph no longer returns data - retain the neighboring graph isolation assertion - prevent false-positive truncate regression coverage
imbajin
left a comment
There was a problem hiding this comment.
TODO: check the diff carefully (inconsistent)
Overview
Important
HStore uses a 2-byte GraphId as the graph-isolation boundary. Before this fix, a new graph whose first write used batch PUT or MERGE could encode data with the reserved missing GraphId
0xFFFE. Multiple graphs could then share the same physical RocksDB key range.Closes #3095. This PR extracts the focused fix from hugegraph#163.
Physical key layout
The GraphId prefix is what keeps data from different graphs in separate physical ranges:
0001 │ shared-key │ codeFFFE │ shared-key │ code0002 │ shared-key │ codeFFFE │ shared-key │ code65534(0xFFFE) is reserved for a missing GraphId mapping. The old batch path calledgetKey(), which returned this sentinel instead of allocating an ID.Trigger conditions
The collision requires all three conditions:
Once multiple graphs meet these conditions in the same partition, they share the
FFFEprefix.Data impact
Note
RocksDB accepts these keys as valid, so the corruption is silent: requests can succeed without exceptions, failed health checks, or Raft errors.
Why this can remain hidden in a long-running deployment
Fix
getKey()getKeyOrCreate()FFFEThe first-allocation path also checks whether the vertex table exists before scanning it. This avoids upgrading the column-family read lock held by the prepared batch into a write path that creates the missing table.
Regression coverage
BatchGraphIsolationTestexercises the realBusinessHandler.doBatch()and RocksDB path:mvn test -pl hugegraph-store/hg-store-test -am \ -P store-core-test -Djacoco.skip=true -ntpScope
This focused change keeps the physical-key format, public APIs, dependencies, configuration, and CI workflow unchanged. It does not repair already affected data or include broader GraphId lifecycle, ID-reuse, or historical-data migration work.
PR checklist